On the limited memory BFGS method for large scale optimization

We study the numerical performance of a limited memory quasi Newton method for large scale optimization which we call the L BFGS method We compare its performance with that of the method developed by Buckley and LeNir which combines cyles of BFGS steps and conjugate direction steps Our numerical tests indicate that the L BFGS method is faster than the method of Buckley and LeNir and is better able to use additional storage to accelerate convergence We show that the L BFGS method can be greatly accelerated by means of a simple scaling We then compare the L BFGSmethod with the partitioned quasi Newton method of Griewank and Toint a The results show that for some problems the partitioned quasi Newton method is clearly superior to the L BFGS method However we nd that for other problems the L BFGS method is very competitive due to its low iteration cost We also study the convergence properties of the L BFGS method and prove global convergence on uniformly convex problems